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Foreword 


This  study  was  conducted  in  response  to  a  request  from  the  Chief  of  Naval  Personnel 
(PERS-23)  to  validate  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  selection 
criteria  for  the  Boiler  Technician  (BT),  Machinist's  Mate  (MM),  and  Engineman  (EN)  ratings  for 
both  the  4-  and  6-year  obligor  (4YO/6YO)  programs  (only  BT  and  MM  have  6YO  programs).  The 
study  was  requested  in  conjunction  with  one  requested  and  completed  for  the  Gas  Turbine  Systems 
Technician  ratings,  Electrical  (GSE)  and  Mechanical  (GSM)  (Held  &  Foley,  1991).  The  BT,  MM 
EN  and  GSE/GSM  ratings  are  all  engineering  ratings.  The  study  was  completed  in  October  1989 
(Held,  1989).  The  recommendations  were  implemented  in  March  1990. 

This  effort  was  sponsored  by  PERS-23  and  funded  by  reimbursable  Work  Unit93WRPS578. 
Results,  which  are  published  at  this  time  for  archival  purposes,  are  intended  for  use  by  the  Bureau 
of  Naval  Personnel,  Navy  school  officials,  and  the  research  community. 


J.  D.  McAFEE 
Captain,  U.S.  Navy 
Commanding  Officer 


J.  SILVERMAN 
Technical  Director  (Acting) 
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Problem 


Summary 


This  study  was  conducted  in  response  to  a  request  from  the  Chief  of  Naval  Personnel 
(PERS-23)  to  validate  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  selection 
criteria  for  the  Boiler  Technician  (BT),  Machinist's  Mate  (MM),  and  Engineman  (EN)  ratings  for 
both  the  4-  and  6-year  obligor  (4YO/6YO)  programs  (only  BT  and  MM  have  6YO  programs).  The 
ASVAB  consists  of  the  following  10  tests:  General  Science  (GS),  Arithmetic  Reasoning  (AR), 
Word  Knowledge  (WK),  Paragraph  Comprehension  (PC),  Numerical  Operations  (NO),  Coding 
Speed  (CS),  Auto  and  Shop  Information  (AS),  Mathematics  Knowledge  (MK),  Mechanical 
Comprehension  (MC),  and  Electronics  Information  (El).  Verbal  (VE)  is  comprised  of  WK  and  PC. 

The  study  was  requested  in  conjunction  with  one  requested  and  completed  for  the  Gas  Turbine 
Systems  Technician  ratings.  Electrical  (GSE)  and  Mechanical  (GSM)  (Held  &  Foley,  1991). 
Because  the  BT,  MM,  EN,  and  GSE/GSM  ratings  are  all  engineering  ratings,  consistent  with  Navy 
consolidation  efforts,  ASVAB  selection  standards  recommended  for  GSE/GSM  were  evaluated  for 
BT,  MM,  and  EN. 

Objectives 

The  objectives  of  this  research  were  to  (1)  validate  the  operational  ASVAB  selector  composites 
against  BT,  MM,  and  EN  “A”  school  performance  measures,  (2)  identify  and  evaluate  alternative 
ASVAB  composites  that  would  be  more  effective  for  determining  qualification  for  “A”  school 
assignment,  and  (3)  determine  a  minimum  qualifying  score  for  the  recommended  selector 
composite  that  would  reduce  “A”  school  attrition  without  significantly  reducing  the  percentage  of 
Navy  recruits  available  for  school  assignment. 

Approach 

Each  of  the  BT,  MM,  and  EN  school  samples  was  randomly  divided  into  a  test  selection  sample 
and  a  holdout  sample.  Two  methods  used  a  multiple  regression  procedure  with  the  test  selection 
sample  to  determine  the  most  valid  ASVAB  selector  composite.  TTie  first.  Method  I,  did  not  correct 
for  restriction  in  range  of  test  scores  that  resulted  from  the  ASVAB  selection,  while  the  second. 
Method  n,  did.  Experimental  composites  identified  from  Methods  I  and  II  and  the  operational 
selector  composite(s)  were  validated  in  the  holdout  sample.  Validities  were  compared  after 
correcting  for  restriction  in  range.  When  replacing  the  operational  composite  was  warranted 
(assessed  from  increase  in  validity  or  expected  improvement  in  the  “A”  school  graduation  rate),  an 
existing  Navy  operational  selector  composite  most  similar  to  the  experimental  composite  was 
evaluated  as  a  candidate  replacement. 

Minimum  qualifying  scores  for  adequate  operational  composites  and  for  proposed 
replacements  were  evaluated  on  the  basis  of  (1)  attrition  rate,  (2)  waiver  rate,  (3)  annual  school 
input  requirement,  (4)  percentage  of  the  recruit  population  qualifying  for  school  selection,  and 
(5)  the  number  of  school  graduates  disqualified  from  school  selection. 
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Results  and  Conclusions 


The  operational  selector  composite,  MK+AS,  was  adequate  for  the  BT,  MM,  and  EN  4YO 
programs,  as  was  the  minimum  qualifying  score,  96. 

The  Electronics  composite,  AR+MK+EI+GS,  the  operational  selector  composite  for  the  GSE/ 
GSM  ratings,  had  higher  validity  than  the  operational  composite,  MK+AS,  for  the  BT  6YO 
program  (validity  was  comparable  for  the  other  operational  composite,  VE+AR).  The  validity  for 
AR+MK+EI+GS  was  slightly  higher  than  the  validities  for  the  two  operational  composites  for  the 
MM  6YO  program.  A  minimum  qualifying  score  of  210,  recommended  for  the  GSE/GSM  6YO 
programs,  was  appropriate  for  the  BT  and  MM  6YO  programs. 

Reconunendations 

The  following  recommendations  are  addressed  to  PERS-23: 

1.  The  BT,  MM,  and  EN  4YO  programs  should  retain  their  operational  selector  composite, 
MK+AS,  and  minimum  qualifying  score,  96,  for  school  selection. 

2.  The  AR+MK+EI+GS  composite  should  replace  VE+AR  and  MK+AS  as  the  operational 
selector  composite  for  the  BT  and  MM  6YO  programs.  The  recommended  minimum  qualifying 
score  for  AR+MK+EI+GS  is  210,  the  same  score  recommended  for  the  GSE/GSM  6YO  programs. 

Expectancy  analyses  should  be  conducted  for  the  BT  and  MM  6YO  programs  after 
AR+MK+EI+GS  is  implemented  and  sufficient  data  become  available  to  determine  if  the  210 
minimum  qualifying  score  is  adequate. 

At  the  time  of  this  study,  plans  for  converting  Navy  ships  from  steam  to  gas  turbine  were 
unclear.  If  the  conversion  takes  place,  BT,  MM,  and  EN  personnel  could  be  retrained  for  the  GS 
ratings.  A  smoother  transition  may  result  if  the  ASVAB  selection  standards  for  the  BT,  MM,  EN, 
and  GS  ratings  are  the  same. 
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Introduction 


Background  and  Problem 

This  study  was  conducted  in  response  to  a  request  from  the  Chief  of  Naval  Personnel 
(PERS-23)  to  validate  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  selection 
criteria  for  the  Boiler  Technician  (BT),  Machinist's  Mate  (MM),  and  Engineman  (EN)  ratings  for 
both  the  4-  and  6-year  obligor  (4YO/6YO)  programs  (only  BT  and  MM  have  6YO  programs). 

The  study  was  requested  in  conjunction  with  one  requested  and  completed  for  the  Gas 
Turbine  Systems  Technician  ratings.  Electrical  (GSE)  and  Mechanical  (GSM)  (Held  &  Foley, 
1991).  The  BT,  MM,  EN,  and  GSE/GSM  ratings  are  all  engineering  ratings.  Because  the  BT,  MM, 
EN,  and  GSE/GSM  ratings  are  all  engineering  ratings,  consistent  with  Navy  consolidation  efforts, 
ASVAB  selection  standards  recommended  for  GSE/GSM  were  evaluated  for  BT,  MM,  and  EN. 

The  ASVAB  consists  of  the  following  10  tests:  General  Science  (GS),  Arithmetic  Reasoning  (AR), 
Word  Knowledge  (WK),  Paragraph  Comprehension  (PC),  Numerical  Operations  (NO),  Coding 
Speed  (CS),  Auto  and  Shop  Information  (AS),  Mathematics  Knowledge  (MK),  Mechanical 
Comprehension  (MC),  and  Electronics  Information  (El).  These  tests,  described  briefly  in  Table  1, 
are  used  by  each  service  in  various  combinations  (composites)  to  select  recruits  into  military 
occupations.  The  Navy  has  11  operational  ASVAB  selector  composites,  listed  in  Table  2. 
Periodically,  studies  are  conducted  to  correlate  ASVAB  composites  with  Class  “A”  school 
performance  measures  to  determine  if  the  school  is  using  the  most  effective  (valid)  operational 
selector  composite.  For  schools  that  have  more  than  one  ASVAB  requirement  (two  selector 
composites  and/or  a  minimum  qualifying  score  for  a  composite  test),  analyses  must  support  their 
use  as  screening  instruments  that  do  not  eliminate  valuable  Navy  talent  from  “A”  school 
assignments. 

Table  3  lists  attrition  rates  for  the  data  used  in  this  study,  which  were  collected  for  the  4-year 
period  from  January  1985  through  December  1988.  Table  3  also  lists  sample  sizes,  waiver  rates 
(percentage  of  students  scoring  below  the  operational  composite  minimum  qualifying  score),  and 
fiscal  year  (FY)  1989  school  input  requirements.  While  attrition  rates  for  the  five  programs  in  this 
validation  study  are  low  to  moderate,  the  Navy  Integrated  Training  Resources  and  Administration 
System  reports  attrition  for  a  recent  1-year  period  for  the  6YO  follow-on  “C”  schools  as  more 
severe  (31%  for  BT;  20%  for  MM).  Therefore,  “C”  school  as  well  as  “A”  school  attrition  rates 
were  considered  in  assessing  BT  and  MM  6YO  ASVAB  selection  standards. 

Objectives 

The  objectives  of  this  research  were  to  (1)  validate  the  operational  ASVAB  selector 
composites  against  BT,  MM,  and  EN  school  performance  measures,  (2)  identify  and  evaluate 
alternative  ASVAB  composites  that  would  be  more  effective  for  determining  qualification  for  “A” 
school  assignment,  and  (3)  determine  a  minimum  qualifying  score  for  the  recommended  selector 
composite  that  would  reduce  “A”  school  attrition  without  significantly  reducing  the  percentage  of 
Navy  recruits  available  for  school  assignment. 


1 


Table  1 


Content  of  ASVAB  Tests 


Test 

Abbreviation 

Description 

General  Science 

GS 

A  25-item  test  of  knowledge  of  the  physical 
(13  items)  and  biological  (12  items) 
sciences- 11  minutes. 

Arithmetic  Reasoning 

AR 

A  30-item  test  of  ability  to  solve  arithmetic 
word  problems— 36  minutes. 

Word  Knowledge^ 

WK 

A  35-item  test  of  knowledge  of  vocabulary, 
using  words  embedded  in  sentences  (11 
items)  and  synonyms  (24  items)— 11 
minutes. 

Paragraph  Comprehension^ 

PC 

A  15-item  test  of  reading  comprehension— 
13  minutes. 

Numerical  Operations 

NO 

A  50-item  speed  test  of  ability  to  add, 
subtract,  multiply,  and  divide  one-  and  two- 
digit  numbers-3  minutes. 

Coding  Speed 

CS 

An  84-item  speed  test  of  ability  to  recognize 
numbers  associated  with  words  from  a 
table— 7  minutes. 

Auto  and  Shop  Information 

AS 

A  25-item  test  of  knowledge  of  automobiles, 
shop  practices,  and  use  of  tools- 11  minutes. 

Mathematics  Knowledge 

MK 

A  25-item  test  of  knowledge  of  algebra, 
geometry,  fractions,  decimals,  and 

exponents— 24  minutes. 

Mechanical  Comprehension 

MC 

A  25-item  test  of  knowledge  of  mechanical 
and  physical  principles— 19  minutes. 

Electronics  Information 

El 

A  20-item  test  of  knowledge  of  electronics, 
radio  and  electrical  principles  and 
information— 9  minutes. 

“Verbal  Score:  VE  =  WK  +  PC  (raw  scores). 
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Table  2 


Navy  Operational  ASVAB  Selector  Comiiosites 


Composite 

Components 

General  Technical 

VE+AR 

Mechanical 

VE+MC+AS 

Electronics 

AR+MK+EI+GS 

Clerical 

VE+NO+CS 

Basic  Electricity  &  Electronics 

ARh2MK4GS 

Engineering 

MK+AS 

Cryptologic  Technician 

VE+AR+NQ+CS 

Hospitalman 

VE+MK-tGS 

Machinery  Repairman 

AR+MC+AS 

Submarine 

VE+AR+MC 

Business/Clericai® 

VE+MK4CS 

Note.  See  Table  1  for  full  test  names. 

‘Student  Testing  Program  composite  implemented  July  1987. 


Table  3 

BT,  MM,  and  EN  Class  ‘*A”  School  Samples 


Program 

Academic 

Attrition  Rate  (%) 
Nonacademic 

Total 

Sample  Size 

Waiver 
Rate®  (%) 

FY89  School 
Input 

Requirement 

BT4YO 

6.3 

6.5 

12.8 

4,5% 

8.3 

1,001 

MM4YO 

6.5 

6.8 

13.3 

6,141 

4.0 

2,192 

EN4YO 

1.9 

4.1 

6.0 

4,356 

10.0 

1,205 

BT6YO 

1.7 

5.6 

7.3 

781 

10.0 

356 

MM6YO 

1.1 

4.2 

5.3 

756 

11.0 

429 

Note.  Data  were  collected  from  January  1985  through  December  1988. 

“Waiver  rate  is  the  percentage  of  students  who  score  below  the  operational  composite  minimum  qualifying  score. 
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Predictors 


Approach 


The  predictors  used  in  this  study  were  the  10  tests  of  ASVAB  Forms  8  through  14,  described 
briefly  in  Table  1.  Raw  test  scores  were  standardized  to  a  mean  of  50  and  a  standard  deviation  of 
10  using  norms  obtained  from  the  American  Youth  Population  (Department  of  Defense,  1982). 

Criterion 

The  criterion  was  final  school  grade  (FSG)  provided  by  the  BT,  MM,  and  EN  “A”  schools 
Although  FSG  is  scaled  from  0  to  100,  passing  scores  are  usually  between  70  and  1(X).  A 
mathematical  procedure  developed  by  Abrahams  and  Alf  (1992)  and  detailed  in  Appendix  A 
estimated  FSGs  for  attrites. 

Samples 

See  Table  3  for  the  BT,  MM,  and  EN  sample  sizes  and  a  breakdown  of  academic  and 
nonacademic  attrition  rates. 

Data  Analyses 

Each  of  the  BT,  MM,  and  EN  school  samples  was  randomly  divided  into  a  test  selection 
sample  (60%  of  the  students)  and  a  holdout  sample  (40%  of  the  students).  Prior  to  this  assignment, 
students  were  sorted  into  graduates,  academic  attrites,  and  nonacademic  atuites  to  ensure  equal 
percentages  were  present  in  the  lest  selection  and  holdout  samples. 

Two  methods  were  used  with  the  test  selection  sample  to  determine  the  ASVAB  composite 
most  predictive  of  FSG.  Both  methods  use  a  forward  stepwise  multiple  regression  procedure  in 
which  a  prediction  equation  starts  with  the  ASVAB  test  that  has  the  highest  correlation  with  FSG, 
followed  by  tests  that  provide  the  largest  increase  in  the  multiple  correlation.^  The  first  four  tests  to 
enter  the  equation  were  designated  as  the  experimental  selector  composite.  Method  I  did  not  correct 
for  restriction  in  range  of  ASVAB  test  scores  resulting  from  ASVAB  selection,  while  Method  n 
did.  The  multivariate  correction  procedure  for  Method  II  is  explained  in  Appendix  B  (Lawley, 
1943)  where  it  is  applied  to  BT  4YO  data.  Results  from  the  multiple  regression  analysis  for 
Methods  I  and  II,  also  for  BT  4YO  data,  are  in  Appendix  C. 

The  most  predictive  composites  identified  by  the  two  methods  and  the  operational  selector 
composite(s)  were  then  cross-validated  in  the  holdout  sample.  Composite  scores  (used  to  correlate 
with  FSG)  were  calculated  by  summing  standardized  test  scores.  This  procedure  unit  weights  each 
tesL^ 

Composite  validities  were  compared  after  correcting  for  restriction  in  range.  Replacing  the 
operational  selector  composite  was  recommended  when  the  experimental  demonstrated  (1)  a  .05 
increase  in  validity  or  (2)  a  2%  improvement  in  the  graduation  rate.^ 


*For  the  multiple  regression,  WK  and  PC  were  combined  into  the  ASVAB  Verbal  (VE)  composite. 

^Unit  weights  generalize  to  cross-validation  samples  more  successfully  than  exact  weights  derived  from  regression 
analysis  (Dawes  &  Corrigan,  1974).  However,  regression  weights  determine  optimal  assignment  in  models  of 
classification  efficiency  (Brogden,  1955). 

^Tbe  Taylor  Russel  tables  (1939)  were  used  to  translate  gain  in  validity  into  expected  gain  in  the  graduation  rate. 


4 


When  replacing  an  operational  composite  was  warranted,  existing  Navy  operational  selector 
composites  (Table  2)  most  similar  to  the  experimental  composite  were  evaluated  as  candidate 
replacements.  The  choice  is  limited  to  Navy  operational  selector  composites  because,  over  the 
course  of  numerous  validation  studies,  implementing  statistically  derived  composites  could  result 
in  an  operationally  unmanageable  number  of  highly  correlated  operational  selector  composites.^ 

Finally,  minimum  qualifying  scores  for  adequate  operational  composites  and  for  proposed 
replacements  were  evaluated  on  the  basis  of  (1)  attrition  rate,  (2)  waiver  rate,  (3)  aimud  school 
input  requirement,  (4)  percentage  of  the  recruit  population  qualifying  for  school  selection,  and 
(5)  the  number  of  school  graduates  disqualified  from  school  selection. 

Results  and  Conclusions 

Experimental  Selector  Composites 

Table  4  lists  the  experimental  composites  identified  by  Methods  I  and  n  for  the  BT,  MM,  and 
EN  programs. 

Table  4 


Experimental  Composites  Identified  for  the  BT,  MM,  and  EN  Test  Selection  Samples 


Rating/Program 

Method  I 

Method  n 

BT4YO 

AR-fEhMK+MC 

< - 

- > 

AR-fQ-fMK+MC 

MM4YO 

AR4MC+MK+EI 

< - 

- > 

AR^C-fMK+H 

EN4YO 

Q-i>AR4GS»MC 

AR-tEI+GS+AS 

BT6YO 

Q-»>MK+MC+VE 

EI-i-AR-fMK+VE 

MM6YO 

MK+MC+GS+CS 

MK+MC+VE+EI 

Not«. 

1.  See  Table  1  for  full  lest  names. 

2.  Arrows  indicaie  that  Methods  I  and  Ilkleniifiedlbe  same  composite. 


Table  S  lists  validities  (uncorrected,  ry;  corrected,  r^)  for  the  operational  and  experimental 
.selector  composites  for  the  BT,  MM  ^d  EN  programs.  Corrected  validities  were  compared  for 
this  study. 

For  the  BT,  MM,  and  EN  4YO  programs,  the  validities  for  the  experimental  composites  were 
between  .02  and  .04  higher  than  the  validities  for  the  operational  composite,  MK+AS.  The  Taylor 


new  composite  could  be  implemented  f(x  the  Navy,  as  was  the  case  of  the  Business/Clerical  composite 
(Table  2),  if  an  experimental  composite  was  consistently  d^ved  for  a  number  of  schods  within  an  occupational 
groiq>  but  was  not  one  of  the  existing  Navy  operational  selector  composites. 


Table  5 


Operational  and  Experimental  Composite  Cross- Validities  for  the 
BT,  MM,  and  EN  Holdout  Samples 


Operational/Experimental  Composite 

Cross- Vilidities® 

fu  Tc 

BT4YO 

MK-i-AS  (Operational) 

.26 

.39 

AR-HEI+MK4-MC  (Experimental-Methods  I  &  II) 

.31 

.43 

MM4YO 

MK+AS  (Operational) 

.29 

.43 

AR+MC+MK+EI  (Experimental-Methods  I  &  II) 

.32 

.45 

EN4YO 

MK-fAS  (Operational) 

.37 

.58 

EI-»-AR4GS-»-MC  (Experimental-Method  I) 

.44 

.62 

AR+EI+GS-fAS  (Experimental-Method  II) 

.42 

.61 

BT6YO 

MK+AS  (Opoational) 

.28 

.56 

VE+AR  (Operational) 

.24 

.55 

EI+MK+MC+VE  (Experimental-Method  I) 

.34 

.59 

EI+AR+MK+VE  (Experimental-Method  II) 

.33 

.59 

MM6YO 

MK+AS  (Operational) 

.38 

.60 

VE+AR  (Operational) 

.29 

.54 

MK+MC+GS-tCS  (Experimental-Method  I) 

.41 

.61 

MK+MC+VE+EI  (Experimental-Method  II) 

.44 

.64 

Note.  See  Table  1  for  full  test  names. 

“Both  Ty  and  r,.  (validities  uncorrected  and  corrected  for  restriction  in  range,  respectively)  are  Person  product-moment 
correlations.  Multivariate  formulas  were  used  for  corrections. 


Russell  tables  (1939)  translate  these  validity  gains  into  less  than  1%  expected  increases  in 
graduation  rates,  which  does  not  warrant  replacing  the  operational  selector  composite  for  the  4YO 
programs. 

For  the  BT  6YO  program,  the  validity  of  .59  for  the  two  experimental  composites, 
EI-hMK-i-MC+VE  and  EI-t-AR+MK-t-VE,  was  .03  higher  than  the  validity  of  .56  for  the  operational 
composite,  MK-hAS,  and  .04  higher  than  the  validity  of  .55  for  the  operational  composite,  VE-t-AR. 
The  .04  validity  increase  translates  into  a  2%  expected  increase  in  the  graduation  rate,  which 
warrants  evaluation  of  candidate  replacement  composites  for  the  BT  6YO  program. 
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For  the  MM  6Y0  program,  the  validity  of  .64  for  the  experimental  composite, 
MK+MC+VE+EI,  was  .04  higher  than  the  validity  of  .60  for  the  operational  composite,  MK+AS, 
and  .10  higher  than  the  validity  of  .54  for  the  operational  composite,  VE+AR.  Both  validity 
increases  translate  into  a  2%  expected  increase  in  the  graduation  rate,  which  warrants  evaluation 
of  candidate  replacement  composites  for  the  MM  6YO  program. 

Candidate  Composite  Selection  and  Evaluation:  BT  and  MM  6YO  Programs 

Of  the  Navy  operational  selector  composites  in  Table  2,  the  Electronics  composite, 
AR+MK+EI+GS,  and  the  Submarine  composite,  VE+AR+MC,  were  selected  as  candidate 
replacement  composites  for  the  BT  and  MM  6YO  programs  because  they  are  most  similar  to  the 
BT  and  MM  6YO  experimental  composites  (from  Table  5,  EI+MK+MC+VE  and 
EI+AR+MK+VE  for  BT  6YO;  MK+MC+GS+CS  and  MK+MC+VE+EI  for  MM  6YO). 

Table  6  lists  the  uncorrected  and  corrected  cross-validities  for  the  candidate  composites  for  the 
BT  and  MM  6YO  holdout  samples. 


Table  6 


Candidate  Composite  Cross- Validities  for  the  BT  and  MM  6YO  Holdout  Samples 


Candidate  Composites 

Cross-Validities^ 

Tu 

fc 

BT6YO 

AR+MK+EI+GS  (Candidate  I) 

.34 

.60 

VE+AR+MC  (Candidate  II) 

.32 

.58 

MM6YO 

AR+MK+EI+GS  (Candidate  I) 

.39 

.61 

VE+AR+MC  (Candidate  D) 

.42 

.62 

Note.  See  Table  1  for  full  lest  names. 

^Both  Ty  and  r^.  (validities  unconected  and  corrected  for  restriction  in  range,  respectively)  are  Pearson  product-moment 
ctxrelations.  Multivariate  formulas  were  used  for  corrections. 


For  BT  6YO  and  MM  6YO,  respectively,  the  validities  of  .60  and  .61  for  the  candidate  I 
composite,  AR+MK+EI+GS,  and  the  validities  of  .58  and  .62  for  the  candidate  11  composite, 
VE+AR+MC,  were  comparable  to  the  validities  for  the  two  programs’  experimental  composites. 
Further  evaluation  suggests  that  AR+MK+EI+GS  should  be  the  proposed  operational  selector 
composite  for  the  two  6YO  programs  because  AR+MK+EI+GS  is  the  only  existing  Navy 
operational  selector  composite  containing  the  El  test,  which  was  found  in  three  of  the  four  6YO 
experimental  composites  (see  Table  4).  Also,  AR+MK+EI+GS  is  the  operational  selector 
composite  for  the  other  engineering  ratings,  GSE  and  GSM. 
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Minimum  Qualifying  Scores:  BT,  MM,  and  EN  4YO  Programs 

For  the  BT,  MM,  and  EN  4Y0  programs,  the  current  operational  selector  composite  with 
minimum  qualifying  score,  MK+AS=96,  qualifies  74%  of  the  recruit  population.  Given  moderate 
attrition  rates  for  the  4YO  programs  and  high  annual  school  input  requirements,  raising  the 
minimum  qualifying  score  cannot  be  justified. 

At  this  time  AR+MK+EI-i<jS,  which  is  appropriate  for  the  BT  and  MM  6YO  programs,  cannot 
be  recommended  for  the  BT,  MM,  and  EN  4YO  programs  because  its  use,  even  with  a  low 
190  minimum  qualifying  score,  would  have  disqualified  30%  of  this  study’s  graduates  from  school 
selection. 

Minimum  Qualifying  Scores:  BT  and  MM  6YO  Programs 

Two  AR+MK+EI+GS  minimum  qualifying  scores  were  evaluated  for  the  BT  and  MM  6YO 
programs.  The  first,  210,  was  recommended  for  the  GSE/GSM  6YO  programs  (Held  &  Foley, 
1991).  The  second,  218,  was  the  minimum  qualifying  score  for  the  GS  programs  at  the  time  of  the 
GSE/GSM  study.  Of  the  two  scores,  210  was  evaluated  as  most  appropriate  because  it 
(1)  disqualified  fewer  BT  and  MM  graduates  from  “A"  school  selection  (for  BT,  210  and 
218  disqualified  78%  and  55%  of  the  graduates,  respectively;  for  MM,  210  and  218  disqualiHed 
90%  and  74%  of  the  graduates,  respectively)  and  (2)  qu^dified  more  recruits  for  “A”  school 
selection  (210  qualifies  43%;  218,  39%).  Also,  218  for  AR+MK+EI+GS  has  already  been 
evaluated  as  too  high  for  the  GSE/GSM  6YO  programs. 

Recommendations 

The  following  recommendations  are  addressed  to  PERS-23: 

1.  The  BT,  MM,  and  EN  4YO  programs  should  retain  their  operational  selector  composite, 
MK+AS,  and  minimum  qualifying  score,  96,  for  school  selection. 

2.  The  AR+MK+EI+GS  composite  should  replace  VE+AR  and  MK+AS  as  the  operational 
selector  composite  for  the  BT  and  MM  6YO  programs.  The  recommended  minimum  qualifying 
score  for  AR+MK+EI+GS  is  2 10,  the  same  score  recommended  for  the  GSE/GSM  6YO  programs. 

Expectancy  analyses  should  be  conducted  for  the  BT  and  MM  6YO  programs  after 
AR+MK+EI+GS  is  implemented  and  sufficient  data  become  available  to  determine  if  the  210 
minimum  qualifying  score  is  adequate. 

At  the  time  of  this  study,  plans  for  converting  Navy  ships  from  steam  to  gas  turbine  were 
unclear.  If  the  conversion  takes  place,  BT,  MM,  and  EN  personnel  could  be  retrained  for  the  GS 
ratings.  A  smoother  transition  may  result  if  the  ASVAB  selection  standards  for  the  BT,  MM,  EN, 
and  GS  ratings  are  the  same. 
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Appendix  A 
Scoring  of  Failures 


A-O 


Scoring  of  Failures 


The  scoring  of  failures  procedure  is  based  on  the  assumption  that,  for  a  population  of  Navy 
applicants,  the  combined  distribution  of  final  school  grade  (FSG)  for  graduates  and  attrites  is 
normal.  On  the  basis  of  the  mathematical  properties  of  a  noimal  curve,  a  mean  FSG  for  attrites  can 
be  calculated  at  the  appropriate  lower  point  of  the  FSG  distribution  given  the  following  values. 

p  =  the  proportion  of  graduates. 

q  =  the  proportion  of  attrites. 

}Cg  =  the  mean  FSG  for  graduates 

SDg  =  the  standard  deviation  of  FSGs  for  graduates 

z  =  the  z-score  (standard  score)  above  which  the  proportion,  p,  falls. 

y  =  the  height  of  the  normal  curve  at  z. 

Step  1 

The  mean  FSG  for  attrites,  X^,  can  be  determined  as  follows; 

Xa  =  Xg  -  A  (SDg),  where  A  = 

Step  2 

Assign  the  estimated  mean  criterion  score  determined  in  step  1  to  each  attrite. 

Step  3 

Compute  the  correlation  between  each  prcuictor  and  the  criterion  for  the  combined  distribution 
of  graduates  and  attrites. 

Step  4 

Correct  the  correlations  from  step  3  for  coarse  grouping  (assigning  a  mean  criterion  score  to 
every  attrite  reduces  variance  and,  therefore,  the  coirelation  coefficient).  The  foimula  used  for  this 
correction  is: 

r^,  =  r^y  /SDz,  where 

SDz'  = 
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Appendix  B 

Correction  Procedure  Used  in  Method  II 
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Correction  Procedure  Used  in  Method  II 


In  order  for  the  regression  analysis  used  to  derive  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB)  composite  most  predictive  of  final  school  grade  (FSG)  not  to  be  biased  against 
tests  used  for  school  selection,  test  scores  must  be  corrected  for  restriction  in  range.  This  is 
accomplished  in  Method  II  by  using  a  Navy  applicant  population  ASVAB/FSG  intercorrelation 
matrix  where  correlations  between  ASVAB  tests  and  FSG  are  estimated  using  multivariate 
correction  formulas  (Lawley,  1943). 

The  next  page  gives  two  intercorrelation  matrices  (including  means  and  standard  deviations) 
required  for  the  multivariate  correction  procedure.  The  first  is  the  ASVAB/FSG  intercorrelation 
matrix  for  the  Boiler  Technician  4-year  obligor  test  selection  sample.  The  second  is  the  ASV^ 
intercorrelation  matrix  for  a  Navy  applicant  population.  The  population  correlations  between 
ASVAB  and  FSG  estimated  by  multivariate  corrections  are  at  the  bottom  of  the  page. 
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Table  B-1 


Required  Multivariate  Matrices  and  Output 


GS  AR  NO 

CS 

AS 

MK 

MC 

El 

VE 

FSG 

Mean 

SD 

BT  4Y0  Test  Selection  Sample  Intercoirelaiions  With  Means  and  Standard  Deviations 

GS 

l.(XX)  .364  -.084 

-.007 

.415 

.326 

.472 

.544 

.688 

.182 

50.42 

4.33 

AR 

1.000  .212 

.183 

.191 

.606 

.459 

.330 

.393 

.229 

50.44 

7.19 

NO 

1.000 

.497 

-.216 

.294 

-.040 

-.111 

-.078 

.070 

52.22 

6.95 

CS 

1.000 

-.095 

.204 

.056 

.034 

.056 

.104 

51.93 

6.91 

AS 

1.000 

-.063 

.474 

.534 

.414 

.125 

55.80 

7.27 

MK 

1.000 

.339 

.232 

.303 

.222 

49.83 

7.13 

MC 

1.000 

.512 

.476 

.214 

52.30 

7.61 

El 

1.000 

.525 

.213 

52.21 

7.66 

VE 

1.000 

.171 

50.72 

6.01 

FSG 

1.000 

83.47 

5.69 

Population  (Applicant  FY87)  Interconelatioas  With  Means  and  Standtird  Deviations 

GS 

1.000  .607  .231 

.228 

.511 

.596 

.648 

.667 

.786 

51.88 

8.48 

AR 

1.000  .452 

.380 

.410 

.751 

.642 

.535 

.634 

51.45 

8.49 

NO 

1.000 

.611 

.033 

.452 

.228 

.144 

.310 

53.12 

7.56 

CS 

1.000 

.048 

.368 

.230 

.166 

.333 

52.72 

7.67 

AS 

1.000 

274 

.629 

.656 

.454 

52.91 

9.14 

MK 

1.000 

.576 

.484 

.562 

51.20 

8.74 

MC 

1.000 

.664 

.604 

53.15 

9.37 

El 

1.000 

.603 

52.12 

9.06 

VE 

1.000 

52.33 

7.02 

FSG 

.276  .401  .300 

.319 

.131 

.430 

.290 

.198 

.324 

Note.  See  Table  1  for  full  test  names. 
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Multiple  Regression  for  Methods  I  and  II 
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Table  C-l 


Multiple  Regression  for  Methods  I  and  II 


TEST 

STEP 

MULTR 

RSQ 

F 

FSIG 

RSQCH 

FCH 

SIGCH 

REG-DF 

RES-DF 

BT  4YO  Test  Selection  Sample  Method  I  (AR+El+MK+MC) 

AR 

1 

.2263 

.0512 

148.702 

.000 

.0512 

148.702 

.000 

1 

2755 

m 

2 

.2697 

.0727 

107.979 

.000 

.0215 

63.863 

.000 

2 

2754 

MK 

3 

.2880 

.0829 

82.990 

.000 

.0102 

30.685 

.000 

3 

2753 

MC 

4 

.2943 

.0866 

65.220 

.000 

.0037 

11.005 

.001 

4 

2752 

Recniit  Applicant  Population  (FY87)  Metlxxl  II  (AR+EI+MK+MC) 

AR 

1 

,3490 

.1218 

.1218 

H 

2 

.3865 

.1494 

.1494 

MK 

3 

,4002 

,1602 

.1602 

MC 

4 

4050 

.1640 

.1640 

Note.  See  Table  1  for  full  test  names. 


The  multiple  regression  results  (SPSSx,  1983)  for  Method  I  show  that  El  is  entered  into  the 
composite  equation  at  step  2,  at  which  point  the  multiple  correlation  for  the  composite  AR+EI  is 
.2697.  The  squared  multiple  correlation  (the  proportion  of  final  school  grade  (FSG)  variance 
accounted  for  by  the  composite)  is  .0727.  The  F  statistic  to  determine  the  significance  of  the 
predictive  relationship  between  the  composite  AR+EI  and  FSG  is  107.979.  The  probability  that 
this  predictive  relationship  is  due  to  chance  is  less  than  .001.  The  change  in  the  squared  multiple 
con'elation  upon  entering  the  El  test  into  the  equation  is  .0215.  The  F  statistic  for  change  (to 
determine  the  significance  of  the  increase  in  the  predictive  relationship  by  adding  El)  is  63.863, 
while  the  probability  that  the  significance  of  this  addition  is  due  to  chance  is  less  than  .001.  The 
regression  and  residual  degrees  of  freedom  are  2  and  2754,  respectively. 

Method  II  is  based  on  conected  correlations.  Since  there  are  no  appropriate  significance  tests 
for  corrected  correlations,  the  F  tests  for  this  method  do  not  apply. 
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